Methods and systems for movement of an automatic cleaning device using video signal

ABSTRACT

A method of cleaning an area using an automatic cleaning device may include receiving, from a video camera, information associated with an edge located on a surface, determining, by an automatic cleaning device, a position of the automatic cleaning device on the surface relative to the edge and using the received information to move the automatic cleaning device from the determined position along a path so that the automatic cleaning device cleans the surface along the path. The path may be substantially parallel to the edge, and the edge may be located a distance from a reference point on the automatic cleaning device during movement of the automatic cleaning device.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is related to U.S. patent application Ser. No.12/616,452 filed Nov. 11, 2009.

BACKGROUND

Mobile robotic devices have minimized the human effort involved inperforming everyday tasks. For example, automatic cleaning devices helpmaintaining and cleaning surfaces, such as hardwood floors, carpet andthe like. Mobile robotic devices are useful, but location detection canbe a challenge for the operation of such devices.

SUMMARY

Before the present methods are described, it is to be understood thatthis invention is not limited to the particular systems, methodologiesor protocols described, as these may vary. It is also to be understoodthat the terminology used herein is for the purpose of describingparticular embodiments only, and is not intended to limit the scope ofthe present disclosure which will be limited only by the appendedclaims.

It must be noted that as used herein and in the appended claims, thesingular forms “a,” “an,” and “the” include plural reference unless thecontext clearly dictates otherwise. Unless defined otherwise, alltechnical and scientific terms used herein have the same meanings ascommonly understood by one of ordinary skill in the art. As used herein,the term “comprising” means “including, but not limited to.”

In an embodiment, a method of cleaning an area using an automaticcleaning device may include receiving, from a video camera, informationassociated with an edge located on a surface, determining, by anautomatic cleaning device, a position of the automatic cleaning deviceon the surface relative to the edge and using the received informationto move the automatic cleaning device from the determined position alonga path so that the automatic cleaning device cleans the surface alongthe path. The path may be substantially parallel to the edge, and theedge may be located a distance from a reference point on the automaticcleaning device during movement of the automatic cleaning device.

In an embodiment, a method of cleaning an environment using an automaticcleaning device may include sending, to a video camera, a request forinformation comprising a color value and a tolerance value, receiving,from the video camera, information associated with an area correspondingto the color value and the tolerance value and using the receivedinformation to adjust a position of the automatic cleaning device on asurface of an environment.

In an embodiment, a method of cleaning an area using an automaticcleaning device may include receiving, from a video camera, informationassociated with an edge located on a surface, determining, by anautomatic cleaning device, a position of the automatic cleaning deviceon the surface relative to the edge; and adjusting the position of theautomatic cleaning device based on a proximity of the automatic cleaningdevice to the edge.

A system for cleaning an area using an automatic cleaning device mayinclude an automatic cleaning device having a processing device and acomputer-readable storage medium; and a video camera in communicationwith the automatic cleaning device. The computer-readable storage mediummay include one or more programming instructions for receiving, from thevideo camera, information associated with an edge located on a surfaceof an area, determining a position of the automatic cleaning device onthe surface relative to the edge, and using the received information tomove the automatic cleaning device from the determined position along apath. The path may be substantially parallel to the edge, and the edgemay be located a distance from a reference point on the automaticcleaning device during movement of the automatic cleaning device.

In an embodiment, a system for cleaning an environment using anautomatic cleaning device may include an automatic cleaning devicehaving a processing device and a computer-readable storage medium, and avideo camera in communication with the automatic cleaning device. Thecomputer-readable storage medium may include one or more programminginstructions for sending, to the video camera, a request for informationcomprising a color value and a tolerance value, receiving, from thevideo camera, information associated with an area corresponding to thecolor value and the tolerance value, and using the received informationto adjust a position of the automatic cleaning device on a surface ofthe environment.

BRIEF DESCRIPTION OF THE DRAWINGS

Aspects, features, benefits and advantages of the present invention willbe apparent with regard to the following description and accompanyingdrawings, of which:

FIG. 1 illustrates an exemplary robotic system according to anembodiment.

FIG. 2 illustrates an exemplary method of navigating a mobile roboticdevice according to an embodiment.

FIG. 3 illustrates an exemplary method of cleaning an area with anautomatic cleaning device according to an embodiment.

FIG. 4 illustrates an exemplary diagram of a mobile robotic device'smovement according to an embodiment.

FIG. 5 illustrates an exemplary method of navigating a mobile roboticdevice according to an embodiment.

FIG. 6 illustrates an exemplary method of navigating an automaticcleaning device according to an embodiment.

DETAILED DESCRIPTION

FIG. 1 illustrates an exemplary robotic system according to anembodiment. As illustrated by FIG. 1, a robotic system may include amobile robotic device 100 and a video camera 105. In an embodiment, themobile robotic device 100 may be an autonomous device that is capable ofautomatically navigating its environment. A mobile robotic device 100may include a processing device, a computer-readable storage mediumand/or the like. In an embodiment, a mobile robotic device 100 may be incommunication with one or more external processing devices,computer-readable storage mediums and/or the like.

In an embodiment, the video camera 105 may be a color video camera, ablack and white video camera and/or the like. In an embodiment, thevideo camera 105 may be mounted on a front portion of a mobile roboticdevice 100. In an alternate embodiment, the video camera 105 may bemounted on a rear portion, a side portion or other portion of the mobilerobotic device 100. The video camera 105 may include a processingdevice, a computer-readable storage medium and/or the like. In anembodiment, the video camera 105 may be in communication with the mobilerobotic device 100 and/or one or more external processing devices,computer-readable storage mediums and/or the like. For example, thevideo camera 105 may collect information about the mobile roboticdevice's surroundings such as coordinates, navigational information,visual information and/or the like. The video camera 105 may providethis information to the mobile robotic device 100.

In an embodiment, the video camera 105 may be a CMUcam3. The CMUcam3 isan ARM7TDMI-based programmable embedded computer vision sensor developedby Carnegie Mellon University. Additional and/or alternate video camerasmay be used within the scope of this disclosure.

FIG. 2 illustrates an exemplary method of navigating a mobile roboticdevice according to an embodiment. As illustrated by FIG. 2, a mobilerobotic device may receive 200 information regarding its surroundingenvironment from a video camera. For example, in an embodiment, a videocamera may perform image processing on a video frame to locate one ormore substantially straight lines in the image. In an embodiment, asubstantially straight line may be representative of a boundary or edgebetween rooms, floor types and/or the like. For example, a video cameramay be able to detect an edge between two adjoining rooms. Similarly, avideo camera may be able to detect an edge between a carpeted floor anda tile floor. In an embodiment, an edge may be detected based on colorcontrast and/or the like.

In an embodiment, the mobile robotic device may receive 200 informationregarding an edge from the video camera. The information may includevalues associated with the slope, the intercept and/or the error of aline detected by the video camera. For example, the slope, interceptand/or error associated with a substantially straight lines from animage of a surface, such as a ground surface, a floor surface or othersurface over which the mobile robotic device is travelling may beprovided to the mobile robotic device. In an embodiment, the videocamera may repetitively send updated information to the mobile roboticdevice. For example, the video camera may send the mobile robotic deviceupdated slope values, intercept values and/or error values associatedwith a detected line several times per second.

In an embodiment, a mobile robotic device may use this information todetermine its position relative to the detected edge. For example, amobile robotic device may compare information about its currentposition, such as its coordinates, heading and/or the like, withinformation received from the video camera regarding the edge. Themobile robotic device may use this comparison to determine its positionrelative to the edge, its distance away from the edge and/or the like.

In an embodiment, a mobile robotic device may detect an edge located inthe direction of the mobile robotic device's movement. For example, amobile robotic device may detect an edge in front of the mobile roboticdevice when the mobile robotic device is moving forward. A mobilerobotic device may detect an edge that is substantially perpendicular toits movement. For example, a mobile robotic device may detect an edgeassociated with a wall or other barrier that the mobile robotic deviceis approaching. In an embodiment, a mobile robotic device may adjust itsmotion based on its proximity to a detected edge. For example, a mobilerobotic device may turn itself around or otherwise change its coursewhen it comes within two feet of the detected edge. Additional and/oralternate proximities may be used within the scope of this disclosure.

In an embodiment, the mobile robotic device may determine 205 whether touse any of the information received from the video camera to adjust itsposition. For example, the mobile robotic device may receive 210positional information from one or more sensors, such as sonar, radarand/or the like. The mobile robotic device may compare 215 theinformation that it receives from its sensors and the information thatit receives from the video camera to information from a map of an areaor environment in which the mobile robotic device is navigating.

In an embodiment, a map may be generated by the mobile robotic deviceand/or another computing device during the mobile robotic device's firstnavigation of the area. For example, the mobile robotic device may bemanually controlled through an area during which time the mobile roboticdevice may generate and store a map of the navigated area. In anembodiment, the mobile robotic device may compare 215 the information itreceives 210 from its sensors and the information it receives from thevideo camera to the digital representation of the map to determine whichinformation to use. In an embodiment, the mobile robotic device may usethe information that has the least amount of error as compared to themobile robotic device's current position. For example, a mobile roboticdevice may determine a current position using a map. It may compare theinformation received from its sensor and the information it receivesfrom its video camera to determine which information has the leastamount of error relative to its current position. The mobile roboticdevice may adjust its position based on the information associated withthe least amount of error.

In an embodiment, the mobile robotic device may use the informationassociated with an edge and its determined position to navigate 220 apath. In an embodiment, the mobile robotic device may navigate 220 apath relative to the edge. For example, the mobile robotic device maynavigate 220 a path that begins at the determined position and that runsparallel to the edge. Alternatively, the mobile robotic device maynavigate 220 a path that begins at the determined position and that isnot parallel to the edge.

In an embodiment, the mobile robotic device may navigate 220 a path suchthat the edge is located a certain distance from a reference point onthe mobile robotic device. The reference point may be a side portion, afront portion, a back portion and/or the like of the mobile roboticdevice. For example, a mobile robotic device may use a detected line asa side registration by tracking the line on the right or left side ofthe mobile robotic device.

In an embodiment, a mobile robotic device may include an automaticcleaning device. An automatic cleaning device may be a mobile roboticdevice that can automatically navigate and clean surfaces, such asfloors. FIG. 3 illustrates an exemplary method of cleaning an area withan automatic cleaning device according to an embodiment.

As described above with respect to a mobile robotic device, an automaticcleaning device may receive 300 information regarding an edge from thevideo camera. The information may include values associated with theslope, the intercept and/or the error of a line detected by the videocamera.

In an embodiment, an automatic cleaning device may use this receivedinformation to determine its position relative to the detected edge. Forexample, an automatic cleaning device may compare information about itscurrent position, such as its coordinates, heading and/or the like, withinformation received from the video camera regarding the edge.

In an embodiment, the automatic cleaning device may determine 305whether to use any of the information received from the video camera toadjust its position. For example, the automatic cleaning device mayreceive 310 positional information from one or more sensors, such assonar, radar and/or the like. The automatic device may compare 315 theinformation that it receives from its sensors and the information thatit receives from the video camera to information from a map of an areain which the automatic cleaning device is navigating.

In an embodiment, the automatic cleaning device may compare 315 theinformation it receives 310 from its sensors and the information itreceives from the video camera to the digital representation of the mapto determine which information to use. In an embodiment, the automaticcleaning device may use the information that has the least amount oferror as compared to the automatic cleaning device's current position.For example, an automatic cleaning device may determine a currentposition using a map. It may compare the information received from itssensor and the information it receives from its video camera todetermine which information has the least amount of error relative toits current position. The automatic cleaning device may adjust itsposition based on the information associated with the least amount oferror.

In an embodiment, the automatic cleaning device may use the informationassociated with an edge and its determined position to navigate 320 apath. The automatic cleaning device may navigate 320 a path relative tothe edge. For example, the automatic cleaning device may navigate 320 apath that begins at the determined position and that runs parallel tothe edge. In an embodiment, the automatic cleaning device may navigate320 the path such that the edge is located a certain distance from areference point on the automatic cleaning device. The reference pointmay be a side portion, a front portion, a back portion and/or the likeof the automatic cleaning device. For example, an automatic cleaningdevice may use a detected line as a side registration by tracking theline on the right or left side of the automatic cleaning device.

In an embodiment, the automatic cleaning device may clean 325 at least aportion of its navigated path. For example, an automatic cleaning devicemay be able to determine one or more characteristics corresponding toits determined position. A characteristic may include, for example, afloor type, such as hard wood, tile, carpet and/or the like. Acharacteristic may include whether the position is in a high-trafficarea, a low-traffic area and/or the like. In an embodiment, an automaticcleaning device may determine an appropriate cleaning method based onthe determined characteristics. For example, the automatic cleaningdevice may vacuum a carpeted area, scrub and/or wax a hardwood or tilearea and/or the like.

FIG. 4 illustrates an exemplary diagram of a mobile robotic device'smovement according to an embodiment. As illustrated by FIG. 4, a mobilerobotic device 400 may track an edge 405 between a carpeted portion of afloor 410 and a tiled portion of a floor 415. In an embodiment, themobile robotic device 400 may move relative to the edge 405. Forexample, as illustrated in FIG. 3, the mobile robotic device 400 maymove along the illustrated path 420, which may run parallel to the edge405.

In an embodiment, a mobile robotic device may navigate its surroundingsbased on its location relative to an area of a particular color. An areamay be a surface area, an item, a landmark and/or the like. For example,a mobile robotic device may navigate an area based on its relativeposition to a green chair located in a room.

FIG. 5 illustrates an exemplary method of navigating a mobile roboticdevice according to an embodiment. As illustrated by FIG. 5, a mobilerobotic device may send 500 an information request to the video camera.The information request may include an indication of a color and atolerance associated with the color. In an embodiment, an indication ofa color may be a color name, a number associated with a color and/or thelike. For example, a mobile robotic device may send 500 an instructionto the video camera asking it to search for the color green. Theinstruction may also include a tolerance value which may instruct thevideo camera as to a range of colors that may be accepted.

In an embodiment, the mobile robotic device may receive 505 informationfrom the video camera. For example, the mobile robotic device mayreceive 505 from the video camera coordinates associated with asubstantially central portion of an area corresponding to the specifiedcolor. The mobile robotic device may also receive 505 coordinates of oneor more corners of a rectangle that encompasses the detected area. Forexample, the video camera may send the mobile robotic device coordinatesof the corners of a bounding box that is just large enough to completelyencompass the detected area. In an embodiment, the mobile robotic devicemay receive a number of pixels of the color that were detected in thearea.

In an embodiment, the mobile robotic device may receive 505 from thevideo camera a numerical value associated with a detected color and/oran associated tolerance range. For example, each color that is detectedby the video camera may be assigned a numerical value. The numericalvalue associated with a detected color may be compared to a numericalvalue associated with the requested color. If the numerical valueassociated with the detected color is within a tolerance range (i.e.,the numerical value associated with the requested color+/−the tolerancevalue), the detected color may be accepted. The mobile robotic devicemay receive from the video camera the value of the detected color and/orthe tolerance range.

In an embodiment, the mobile robotic device may determine 510 whether touse any of the information received from the video camera in navigatingits path. For example, the mobile robotic device may receive 515positional information from one or more sensors, such as sonar, radarand/or the like. In an embodiment, the mobile robotic device may compare520 the information that it receives from its sensors and theinformation that it receives from the video camera to information from amap of an area in which the mobile robotic device is navigating.

In an embodiment, a map may be generated by the mobile robotic deviceand/or another computing device during the mobile robotic device's firstnavigation of the area. For example, the mobile robotic device may bemanually controlled through an area during which time the mobile roboticdevice may generate and store a map of the navigated area. In analternate embodiment, a map of an area may be generated manually. Forexample, a may be generated by a user and a digital representation ofthe map may be stored electronically.

In an embodiment, the mobile robotic device may compare 520 theinformation it receives 515 from its sensors and the information itreceives from the video camera to the digital representation of the mapto determine which information to use. In an embodiment, the mobilerobotic device may use the information that has the least amount oferror as compared to the mobile robotic device's current position. Forexample, a mobile robotic device may determine a current position usinga map. It may compare the information received from its sensor and theinformation it receives from its video camera to determine whichinformation has the least amount of error relative to its currentposition. The mobile robotic device may adjust its position based on theinformation associated with the least amount of error.

In an embodiment, the mobile robotic device may adjust 525 its positionbased on the most accurate information. The mobile robotic device mayuse the information to adjust 525 its position, heading and/or the like.For example, an estimated position of the mobile robotic device may bedetermined. The estimated position may include coordinates, a headingand/or the like. In an embodiment, an actual position may be determinedbased on the information associated with the detected area that isreceived by the mobile robotic device. The estimated position may becompared to the actual position, and the mobile robotic device mayadjust 525 its position to compensate for any positional error that mayexist. For example, the mobile robotic device may navigate from itsactual position to the estimated position.

FIG. 6 illustrates an exemplary method of navigating an automaticcleaning device according to an embodiment. As illustrated by FIG. 6, anautomatic cleaning device may send 600 an information request to thevideo camera. The information request may include an indication of acolor and a tolerance associated with the color. The instruction mayalso include a tolerance value which may instruct the video camera as toa range of colors that may be accepted.

In an embodiment, the automatic cleaning device may receive 605information from the video camera. For example, the automatic cleaningdevice may receive 605 from the video camera coordinates associated witha substantially central portion of an area corresponding to thespecified color. The automatic cleaning device may also receive 605coordinates of one or more corners of a rectangle that encompasses thedetected area. In an embodiment, the automatic cleaning device mayreceive a number of pixels of the color that were detected in the area.

In an embodiment, the automatic cleaning device may receive 605 from thevideo camera a numerical value associated with a detected color and/oran associated tolerance range. For example, each color that is detectedby the video camera may be assigned a numerical value. The numericalvalue associated with a detected color may be compared to a numericalvalue associated with the requested color. If the numerical valueassociated with the detected color is within a tolerance range (i.e.,the numerical value associated with the requested color+/−the tolerancevalue), the detected color may be accepted. The automatic cleaningdevice may receive from the video camera the value of the detected colorand/or the tolerance range.

In an embodiment, the automatic cleaning device may determine 610whether to use any of the information received from the video camera innavigating its path. For example, the automatic cleaning device mayreceive 615 positional information from one or more sensors, such assonar, radar and/or the like. In an embodiment, the automatic cleaningdevice may compare 620 the information that it receives from its sensorsand the information that it receives from the video camera toinformation from a map of an area in which the automatic cleaning deviceis navigating.

In an embodiment, a map may be generated by the automatic cleaningdevice and/or another computing device during the automatic cleaningdevice's first navigation of the area. For example, the automaticcleaning device may be manually controlled through an area during whichtime the mobile robotic device may generate and store a map of thenavigated area. In an embodiment, the mobile robotic device may compare620 the information it receives 615 from its sensors and the informationit receives from the video camera to the digital representation of themap to determine which information to use.

In an embodiment, the automatic cleaning device may use the informationthat has the least amount of error as compared to the automatic cleaningdevice's current position. For example, an automatic cleaning device maydetermine a current position using a map. It may compare the informationreceived from its sensor and the information it receives from its videocamera to determine which information has the least amount of errorrelative to its current position. The automatic cleaning device mayadjust its position based on the information associated with the leastamount of error.

In an embodiment, the automatic cleaning device may adjust 625 itsposition based on the most accurate information. The automatic cleaningdevice may use the information to adjust 625 its position, headingand/or the like. For example, an estimated position of the automaticcleaning device may be determined. The estimated position may includecoordinates, a heading and/or the like. In an embodiment, an actualposition may be determined based on the information associated with thedetected area that is received by the automatic cleaning device. Theestimated position may be compared to the actual position, and theautomatic cleaning device may adjust 625 its position to compensate forany positional error that may exist. For example, the automatic cleaningdevice may navigate from its actual position to the estimated position.

In an embodiment, the automatic cleaning device may clean 630 at least aportion of its navigated path. For example, an automatic cleaning devicemay be able to determine one or more characteristics corresponding toits determined position. A characteristic may include, for example, afloor type, such as hard wood, tile, carpet and/or the like. Acharacteristic may include whether the position is in a high-trafficarea, a low-traffic area and/or the like. In an embodiment, an automaticcleaning device may determine an appropriate cleaning method based onthe determined characteristics. For example, the automatic cleaningdevice may vacuum a carpeted area, scrub and/or wax a hardwood or tilearea and/or the like.

It will be appreciated that various of the above-disclosed and otherfeatures and functions, or alternatives thereof, may be desirablycombined into many other different systems or applications. Also thatvarious presently unforeseen or unanticipated alternatives,modifications, variations or improvements therein may be subsequentlymade by those skilled in the art which are also intended to beencompassed by the following claims.

What is claimed is:
 1. A system for cleaning an environment using anautomated cleaning device, the system comprising: an automatic cleaningdevice comprising a processing device and a computer-readable storagemedium; and a video camera in communication with the automatic cleaningdevice; wherein the computer-readable storage medium comprisesprogramming instructions for: generating, by the processing device, adigital representation of a map of an area navigated by the automaticcleaning device, sending, to the video camera, one or more instructionsto search for a color in an area having a color value that is within atolerance range, receiving, from the video camera, informationassociated with at least a portion of the area that corresponds to thecolor value, receiving, by the processing device, position data for theautomatic cleaning device from one or more sensors associated with theautomatic cleaning device, comparing, by the processing device, each ofthe received information and the received position data to the digitalrepresentation of the map to determine which of the received informationand the received position data has a least amount of error relative to acurrent position of the automatic cleaning device, using the receivedinformation to adjust a current position of the automatic cleaningdevice if the received information has the least amount of error, andcleaning at least a portion of a path along which the automatic cleaningdevice is navigated.
 2. The system of claim 1, wherein the programminginstructions for receiving information comprise one or more programminginstructions for receiving one or more of the following: coordinatesassociated with a substantially center portion of the area; coordinatesassociated with one or more corners of a rectangle that surrounds thearea; a number of pixels of the color within the area; and a numericalvalue associated with a detected color.
 3. The system of claim 1,wherein the programming instructions for using the received informationcomprise one or more programming instructions for: adjusting the pathbased on the received information in response to an amount of errorassociated with the received information as compared to the currentposition being less than an amount of error associated with the receivedposition data as compared to the current position.
 4. The system ofclaim 1, wherein the programming instructions for using the receivedinformation to adjust a current position of the automatic cleaningdevice comprise one or more programming instructions for: determining anestimated location of the automatic cleaning device; using the receivedinformation to determine an actual location of the automatic cleaningdevice; and if the estimated location differs from the actual location,moving to the estimated location.
 5. The system of claim 1, wherein theprogramming instructions for cleaning at least a portion of the pathcomprise one or more programming instructions for: determining one ormore characteristics associated with the portion of the surface;determining a cleaning method based on the determined one or morecharacteristics; and applying the cleaning method to the portion of thesurface.